feat: Start Botproject runtime in bot folder #1672

VanyLaw · 2019-11-28T12:54:05Z

Description

This PR is for starting runtime by csharp botproject in bot folder rather than by common botproject.

Solution

When a call to /connect api. Check whether the port 3979 has been used. If port is valid, use it as the endpoint to start runtime, otherwise use another valid port instead.
Check whether the .net core version >=3.0, init user secret and build runtime.
Stop previous bot runtime and start current bot runtime.

Task Item

related to #1516
close #483

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Code refactor (non-breaking change which improve code quality, clean up, add tests, etc)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Doc update (document update)

Checklist

I have added tests that prove my fix is effective or that my feature works
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have functionally tested my change

Screenshots

Please include screenshots or gifs if your PR include UX changes.

github-actions · 2019-11-28T13:03:28Z

Coverage decreased (-0.08%) to 39.937% when pulling 6b04384 on wenyluo/csConnector into f492bef on master.

BotProject/Templates/CSharp/Scripts/build_runtime.ps1

Composer/packages/server/src/models/connector/csharpBotConnector.ts

compulim

I am seeing a lot of coalesce of SensitiveProperties.

You can also consider coalesce at earlier code to reduce these coalesces.

Composer/packages/client/src/store/reducer/index.ts

Composer/packages/server/src/models/settings/defaultSettingManager.ts

Composer/packages/client/src/store/action/setting.ts

Composer/packages/server/src/models/connector/csharpBotConnector.ts

Composer/packages/client/src/store/action/setting.ts

Composer/packages/server/src/controllers/connector.ts

Composer/packages/server/src/models/connector/csharpBotConnector.ts

Composer/packages/client/src/store/action/bot.ts

Composer/packages/server/src/controllers/connector.ts

Composer/packages/server/src/models/connector/csharpBotConnector.ts

Composer/packages/server/src/models/settings/fileSettingManager.ts

VanyLaw · 2019-12-20T16:28:46Z

We shouldn't combine dockerfile for Composer and C# runtime because we'll have multiple languages for runtimes. If we need to have 1 container which runs both (which I think we should avoid), then we should build the composer container, then have the dotnet one extend the base composer container. That pattern would work for other language as well.

Hi @christopheranderson , Is there a solution for the "dockerfile spliting" problem? Like I said before, I have a build error when using two dockerfile, so, Is that ok to use multi-stage instead?

cwhitten · 2020-01-06T23:42:58Z

@VanyLaw have any commands changed to make this work? I'm seeing errors locally, but may be in a bad state:

~/src/BotFramework-Composer/Composer (wenyluo/csConnector) $ docker-compose build
Building composer
Step 1/27 : FROM node:12-alpine as build
 ---> 69c8cc9212ec
Step 2/27 : WORKDIR /src/Composer
 ---> Using cache
 ---> 71bb946b33f5
Step 3/27 : COPY . .
 ---> Using cache
 ---> 7f2deaf0fbd6
Step 4/27 : RUN yarn install --frozen-lock-file
 ---> Using cache
 ---> 2c35707c2fe0
Step 5/27 : ENV NODE_OPTIONS "--max-old-space-size=4096"
 ---> Using cache
 ---> f9d9d45b2d56
Step 6/27 : ENV NODE_ENV "production"
 ---> Using cache
 ---> d9984da779af
Step 7/27 : RUN yarn build:prod
 ---> Running in 45d73df91d2d
yarn run v1.19.1
$ yarn build:dev && yarn build:server && yarn build:client
$ yarn build:tools && yarn build:lib && yarn build:extensions
$ yarn workspace @bfc/tools build:all
$ concurrently --kill-others-on-fail "yarn:build:language-servers"
[build:language-servers] $ cd language-servers && yarn build
[build:language-servers] $ yarn build:lg
[build:language-servers] $ cd language-generation && yarn build
[build:language-servers] $ tsc --build tsconfig.build.json
[build:language-servers] yarn run build:language-servers exited with code 0
$ yarn workspace @bfc/libs build:all
$ yarn build:indexers && yarn build:shared && yarn build:code-editor
$ cd indexers && yarn build
$ yarn clean && yarn build:css && yarn build:ts
$ rimraf lib demo/dist
$ copyfiles --up 1 "src/**/*.css" "src/**/*.scss" lib
$ tsc --build tsconfig.build.json
$ cd shared && yarn build
$ yarn clean && yarn build:css && yarn build:ts
$ rimraf lib demo/dist
$ copyfiles --up 1 "src/**/*.css" "src/**/*.scss" lib
$ tsc --build tsconfig.build.json
$ cd code-editor && yarn build
$ yarn clean && yarn build:css && yarn build:ts
$ rimraf lib demo/dist
$ copyfiles --up 1 "src/**/*.css" "src/**/*.scss" lib
$ tsc --build tsconfig.build.json
$ yarn workspace @bfc/extensions build:all
$ concurrently --kill-others-on-fail "yarn:build:obiformeditor" "yarn:build:visual-designer"
[build:obiformeditor] $ cd obiformeditor && yarn build
[build:visual-designer] $ cd visual-designer && yarn build
[build:obiformeditor] $ yarn clean && yarn build:css && yarn build:ts
[build:visual-designer] $ yarn clean && yarn build:css && yarn build:ts
[build:obiformeditor] $ rimraf lib demo/dist
[build:visual-designer] $ rimraf lib demo/dist
[build:obiformeditor] $ copyfiles --up 1 "src/**/*.css" "src/**/*.scss" lib
[build:visual-designer] $ copyfiles --up 1 "src/**/*.css" "src/**/*.scss" lib
[build:visual-designer] $ tsc --build tsconfig.build.json
[build:obiformeditor] $ tsc --build tsconfig.build.json
[build:visual-designer] yarn run build:visual-designer exited with code 0
[build:obiformeditor] yarn run build:obiformeditor exited with code 0
$ yarn workspace @bfc/server build
$ tsc -p tsconfig.build.json
$ yarn workspace @bfc/client build
$ node scripts/build.js
Creating an optimized production build. This may take a few minutes..
error Command failed with signal "SIGKILL".
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed.
Exit code: 1
Command: /usr/local/bin/node
Arguments: /opt/yarn-v1.19.1/lib/cli.js build
Directory: /src/Composer/packages/client
Output:

info Visit https://yarnpkg.com/en/docs/cli/workspace for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
ERROR: Service 'composer' failed to build: The command '/bin/sh -c yarn build:prod' returned a non-zero code: 1

boydc2014 · 2020-01-07T09:24:46Z

@VanyLaw have any commands changed to make this work? I'm seeing errors locally, but may be in a bad state:

~/src/BotFramework-Composer/Composer (wenyluo/csConnector) $ docker-compose build
Building composer
Step 1/27 : FROM node:12-alpine as build
 ---> 69c8cc9212ec
Step 2/27 : WORKDIR /src/Composer
 ---> Using cache
 ---> 71bb946b33f5
Step 3/27 : COPY . .
 ---> Using cache
 ---> 7f2deaf0fbd6
Step 4/27 : RUN yarn install --frozen-lock-file
 ---> Using cache
 ---> 2c35707c2fe0
Step 5/27 : ENV NODE_OPTIONS "--max-old-space-size=4096"
 ---> Using cache
 ---> f9d9d45b2d56
Step 6/27 : ENV NODE_ENV "production"
 ---> Using cache
 ---> d9984da779af
Step 7/27 : RUN yarn build:prod
 ---> Running in 45d73df91d2d
yarn run v1.19.1
$ yarn build:dev && yarn build:server && yarn build:client
$ yarn build:tools && yarn build:lib && yarn build:extensions
$ yarn workspace @bfc/tools build:all
$ concurrently --kill-others-on-fail "yarn:build:language-servers"
[build:language-servers] $ cd language-servers && yarn build
[build:language-servers] $ yarn build:lg
[build:language-servers] $ cd language-generation && yarn build
[build:language-servers] $ tsc --build tsconfig.build.json
[build:language-servers] yarn run build:language-servers exited with code 0
$ yarn workspace @bfc/libs build:all
$ yarn build:indexers && yarn build:shared && yarn build:code-editor
$ cd indexers && yarn build
$ yarn clean && yarn build:css && yarn build:ts
$ rimraf lib demo/dist
$ copyfiles --up 1 "src/**/*.css" "src/**/*.scss" lib
$ tsc --build tsconfig.build.json
$ cd shared && yarn build
$ yarn clean && yarn build:css && yarn build:ts
$ rimraf lib demo/dist
$ copyfiles --up 1 "src/**/*.css" "src/**/*.scss" lib
$ tsc --build tsconfig.build.json
$ cd code-editor && yarn build
$ yarn clean && yarn build:css && yarn build:ts
$ rimraf lib demo/dist
$ copyfiles --up 1 "src/**/*.css" "src/**/*.scss" lib
$ tsc --build tsconfig.build.json
$ yarn workspace @bfc/extensions build:all
$ concurrently --kill-others-on-fail "yarn:build:obiformeditor" "yarn:build:visual-designer"
[build:obiformeditor] $ cd obiformeditor && yarn build
[build:visual-designer] $ cd visual-designer && yarn build
[build:obiformeditor] $ yarn clean && yarn build:css && yarn build:ts
[build:visual-designer] $ yarn clean && yarn build:css && yarn build:ts
[build:obiformeditor] $ rimraf lib demo/dist
[build:visual-designer] $ rimraf lib demo/dist
[build:obiformeditor] $ copyfiles --up 1 "src/**/*.css" "src/**/*.scss" lib
[build:visual-designer] $ copyfiles --up 1 "src/**/*.css" "src/**/*.scss" lib
[build:visual-designer] $ tsc --build tsconfig.build.json
[build:obiformeditor] $ tsc --build tsconfig.build.json
[build:visual-designer] yarn run build:visual-designer exited with code 0
[build:obiformeditor] yarn run build:obiformeditor exited with code 0
$ yarn workspace @bfc/server build
$ tsc -p tsconfig.build.json
$ yarn workspace @bfc/client build
$ node scripts/build.js
Creating an optimized production build. This may take a few minutes..
error Command failed with signal "SIGKILL".
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed.
Exit code: 1
Command: /usr/local/bin/node
Arguments: /opt/yarn-v1.19.1/lib/cli.js build
Directory: /src/Composer/packages/client
Output:

info Visit https://yarnpkg.com/en/docs/cli/workspace for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
ERROR: Service 'composer' failed to build: The command '/bin/sh -c yarn build:prod' returned a non-zero code: 1

Looks highly likely a memory issue, can you check did you alloc enough memory to docker-compose? In windows, the correct way to do that is

you need to make sure your memory is set to more than 4GB, the value we set on node.js limitation.

VanyLaw · 2020-01-07T12:42:29Z

Hi @cwhitten, The error was throw when client build, it seems like a out of memory issue, maybe extending the memory for docker can fix it.

a-b-r-o-w-n

Composer/packages/server/ComposerDialogs/settings/appsettings.json keeps getting created when launching or restarting the bot. Not sure what is happening.

a-b-r-o-w-n · 2020-01-08T21:36:31Z

Other than that one thing, this is looking good to me.

VanyLaw · 2020-01-09T09:27:07Z

Composer/packages/server/ComposerDialogs/settings/appsettings.json keeps getting created when launching or restarting the bot. Not sure what is happening.

Hi @a-b-r-o-w-n , I debug and find it's because I get the localStorage setting in sync api. but EnvironmentProvider.getCurrent() return the default environment which used null basePath, instead of the current botproject environment.

So I used BotProjectService.getCurrentBotProject()?.environment instead to fix it.

stale

@lei9444

### 01-27-2020 #### Added - feat: enrich prebuilt memory paths and add user defined memory paths ([#1868](#1868)) ([@lei9444](https://github.com/lei9444)) - feat: Language-Understanding LSP ([#1711](#1711)) ([@cosmicshuai](https://github.com/cosmicshuai)) - feat: lu shell api ([#1810](#1810)) ([@zhixzhan](https://github.com/zhixzhan)) - feat: Start Botproject runtime in bot folder ([#1672](#1672)) ([@VanyLaw](https://github.com/VanyLaw)) - feat: Schema-driven visual editor - migrates 15 simple types ([#1811](#1811)) ([@yeze322](https://github.com/yeze322)) - feat: Update OnConversationUpdate actions ([#1794](#1794)) ([@luhan2017](https://github.com/luhan2017)) - feat: Update LG and built in expression functions in LSP ([#1790](#1790)) ([@Danieladu](https://github.com/Danieladu)) - feat: Add dynamic choices to Choice Prompt ([#1777](#1777)) ([@tdurnford](https://github.com/tdurnford)) - feat: update ui for new schema ([#1775](#1775)) ([@alanlong9278](https://github.com/alanlong9278)) - feat: lu all up view ux enhancement ([#1879](#1879)) ([@zhixzhan](https://github.com/zhixzhan)) - feat: add autoClosingPairs for lg editor ([#1789](#1789)) ([@cosmicshuai](https://github.com/cosmicshuai)) #### Fixed - fix: update the lg content to store when editor editing ([#1884](#1884)) ([@lei9444](https://github.com/lei9444)) - fix: Aligned elements in prompt settings ([#1893](#1893)) ([@tdurnford](https://github.com/tdurnford)) - fix: remove label from validations ([#1891](#1891)) ([@tdurnford](https://github.com/tdurnford)) - fix: Undo / redo behavior on LG resources ([#1813](#1813)) ([@lei9444](https://github.com/lei9444)) - fix: botproject build script powershell version limit ([#1857](#1857)) ([@VanyLaw](https://github.com/VanyLaw)) - fix: Add support of auto suggestion memory variable ([#1819](#1819)) ([@cosmicshuai](https://github.com/cosmicshuai)) - fix: Fixed Multiple Choice Type undo/redo functionality ([#1844](#1844)) ([@tdurnford](https://github.com/tdurnford)) - fix: RepeatDialog not shown ([#1835](#1835)) ([@yeze322](https://github.com/yeze322)) - fix: do not preset name when creating a dialog ([#1805](#1805)) ([@liweitian](https://github.com/liweitian)) - fix: luis authoringRegion not update in popup form ([#1818](#1818)) ([@VanyLaw](https://github.com/VanyLaw)) - fix: Check all expressions in dialog ([#1798](#1798)) ([@lei9444](https://github.com/lei9444)) - fix: TextWidget hover ([#1825](#1825)) ([@tdurnford](https://github.com/tdurnford)) - fix: Create bot from scratch may create a non-empty bot ([#1796](#1796)) ([@liweitian](https://github.com/liweitian)) - fix: Updated release version on about page ([#1788](#1788)) ([@tdurnford](https://github.com/tdurnford)) - fix: Remove the unused lufiles in notifications ([#1760](#1760)) ([@lei9444](https://github.com/lei9444)) - fix: update form correctly when undo or redo changes ([#1808](#1808)) ([@lei9444](https://github.com/lei9444)) - fix: handle multiple ctrl+z in lg editor ([#1804](#1804)) ([@zhixzhan](https://github.com/zhixzhan)) #### Changed - refactor: re-provide `getLgTemplates` api ([#1746](#1746)) ([@yeze322](https://github.com/yeze322)) - style: updated delete icon ([#1892](#1892)) ([@tdurnford](https://github.com/tdurnford)) - refactor: let ElementWrapper control the focus state ([#1880](#1880)) ([@yeze322](https://github.com/yeze322)) - refactor: render Elements with composition strategy (ElementRenderer -> ElementWrapper) ([#1873](#1873)) ([@yeze322](https://github.com/yeze322)) #### Other - chore: migrate IfCondition, SwitchCondition, Foreach(Page) to uischema ([#1899](#1899)) ([@yeze322](https://github.com/yeze322)) - chore: migrate TextInput and all other \*Input types to uischema ([#1874](#1874)) ([@yeze322](https://github.com/yeze322)) - chore: migrate SendActivity, BeginDialog, ReplaceDialog to uischema ([#1840](#1840)) ([@yeze322](https://github.com/yeze322)) - chore: retire 'DefaultRenderer' in visual editor ([#1836](#1836)) ([@yeze322](https://github.com/yeze322)) - chore: update lu/lg all up view ([#1806](#1806)) ([@liweitian](https://github.com/liweitian)) - chore: share server's resource to lsp server ([#1793](#1793)) ([@zhixzhan](https://github.com/zhixzhan)) - chore: refactor lg page route & url ([#1756](#1756)) ([@zhixzhan](https://github.com/zhixzhan)) - chore: convert ludown to bf-lu ([#1608](#1608)) ([@lei9444](https://github.com/lei9444)) - chore: proxy lsp ws request in dev ([#1754](#1754)) ([@zhixzhan](https://github.com/zhixzhan)) - docs: fix a typo ([#1901](#1901)) ([@HiltonGiesenow](https://github.com/HiltonGiesenow)) - docs: use correct link in events and triggers documentation ([#1832](#1832)) ([@arafattehsin](https://github.com/arafattehsin)) - docs: fix memory documentation link and typo ([#1833](#1833)) ([@arafattehsin](https://github.com/arafattehsin)) - docs: fix the links to intents and advanced definitions ([#1854](#1854)) ([@arafattehsin](https://github.com/arafattehsin)) - chore: bump immer to 5.2.0 ([#1846](#1846)) ([@cwhitten](https://github.com/cwhitten)) - chore: update botproject packages ([#1809](#1809)) ([@luhan2017](https://github.com/luhan2017)) - chore: update github templates ([@a-b-r-o-w-n](https://github.com/a-b-r-o-w-n)) - chore: update schema and samples ([@luhan2017](https://github.com/luhan2017))

VanyLaw added 4 commits November 28, 2019 16:54

start runtime from samplpebot csharp runtime

69ae226

fix

1f3b631

fix

b8670c9

fix

6b04384

VanyLaw requested review from a-b-r-o-w-n, boydc2014, cwhitten and luhan2017 as code owners November 28, 2019 12:54

VanyLaw changed the title ~~Start Botproject runtime in bot folder~~ [feat]Start Botproject runtime in bot folder Nov 28, 2019

VanyLaw changed the title ~~[feat]Start Botproject runtime in bot folder~~ feat: Start Botproject runtime in bot folder Nov 28, 2019

VanyLaw and others added 2 commits November 28, 2019 22:16

use logger instead of console log

1e0ca12

Merge branch 'master' into wenyluo/csConnector

fbf2ed7

luhan2017 reviewed Nov 29, 2019

View reviewed changes

BotProject/Templates/CSharp/Scripts/build_runtime.ps1 Show resolved Hide resolved

luhan2017 reviewed Nov 29, 2019

View reviewed changes

Composer/packages/server/src/models/connector/csharpBotConnector.ts Show resolved Hide resolved

luhan2017 and others added 3 commits December 2, 2019 10:09

Merge branch 'master' into wenyluo/csConnector

1b17984

fix comment

3964fd7

fix comment

c0f1411

compulim reviewed Dec 3, 2019

View reviewed changes

a-b-r-o-w-n requested changes Dec 3, 2019

View reviewed changes

VanyLaw added 3 commits December 3, 2019 10:29

Merge remote-tracking branch 'origin/master' into wenyluo/csConnector

0334bb8

add port usable validation and use log instead console log

306a0d2

Merge remote-tracking branch 'origin/master' into wenyluo/csConnector

99a2882

a-b-r-o-w-n requested changes Dec 3, 2019

View reviewed changes

VanyLaw added 5 commits December 4, 2019 17:50

fix comments

14c4a21

Merge remote-tracking branch 'origin/master' into wenyluo/csConnector

4d989e6

move build script from template to server

543785b

Merge remote-tracking branch 'origin/master' into wenyluo/csConnector

396e7c7

polish output error message

1cc7e70

a-b-r-o-w-n requested changes Dec 5, 2019

View reviewed changes

VanyLaw dismissed a-b-r-o-w-n’s stale review via 4e1bfb5 December 19, 2019 06:31

VanyLaw added 4 commits December 19, 2019 21:21

split dockerfile into two

1090b93

Merge remote-tracking branch 'origin/master' into wenyluo/csConnector

0bf8040

Merge remote-tracking branch 'origin/master' into wenyluo/csConnector

861d57e

use nultistage instead of two dockerfile

a791621

VanyLaw requested a review from christopheranderson December 21, 2019 06:33

VanyLaw added 2 commits December 27, 2019 16:21

Merge remote-tracking branch 'origin/master' into wenyluo/csConnector

6a09761

Merge remote-tracking branch 'origin/master' into wenyluo/csConnector

3f7d035

hibrenda mentioned this pull request Dec 31, 2019

Problem with running Composer created project from VS2017 #1816

Closed

Merge branch 'master' into wenyluo/csConnector

9b9e139

Merge remote-tracking branch 'origin/master' into wenyluo/csConnector

243f9c8

VanyLaw and others added 5 commits January 7, 2020 20:43

update yarn lock

803fe7a

Merge remote-tracking branch 'origin/master' into wenyluo/csConnector

20036eb

Merge branch 'master' into wenyluo/csConnector

d4be796

Merge branch 'master' into wenyluo/csConnector

551c4a2

add more debug logs

6ce7143

a-b-r-o-w-n requested changes Jan 8, 2020

View reviewed changes

fix settings file mis-created in server folder

429886e

a-b-r-o-w-n approved these changes Jan 9, 2020

View reviewed changes

a-b-r-o-w-n merged commit 89faea0 into master Jan 9, 2020

a-b-r-o-w-n deleted the wenyluo/csConnector branch January 9, 2020 17:55

a-b-r-o-w-n mentioned this pull request Jan 27, 2020

release: 01-27-2020 #1910

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Start Botproject runtime in bot folder #1672

feat: Start Botproject runtime in bot folder #1672

VanyLaw commented Nov 28, 2019 •

edited

Loading

github-actions bot commented Nov 28, 2019

compulim left a comment

VanyLaw commented Dec 20, 2019 •

edited

Loading

cwhitten commented Jan 6, 2020

boydc2014 commented Jan 7, 2020

VanyLaw commented Jan 7, 2020

a-b-r-o-w-n left a comment

a-b-r-o-w-n commented Jan 8, 2020

VanyLaw commented Jan 9, 2020 •

edited

Loading

feat: Start Botproject runtime in bot folder #1672

feat: Start Botproject runtime in bot folder #1672

Conversation

VanyLaw commented Nov 28, 2019 • edited Loading

Description

Solution

Task Item

Type of change

Checklist

Screenshots

github-actions bot commented Nov 28, 2019

compulim left a comment

Choose a reason for hiding this comment

VanyLaw commented Dec 20, 2019 • edited Loading

cwhitten commented Jan 6, 2020

boydc2014 commented Jan 7, 2020

VanyLaw commented Jan 7, 2020

a-b-r-o-w-n left a comment

Choose a reason for hiding this comment

a-b-r-o-w-n commented Jan 8, 2020

VanyLaw commented Jan 9, 2020 • edited Loading

VanyLaw commented Nov 28, 2019 •

edited

Loading

VanyLaw commented Dec 20, 2019 •

edited

Loading

VanyLaw commented Jan 9, 2020 •

edited

Loading